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ABSTRACT 

The need to improve the quality and 
accuracy of cost estimates of proposed new' 
aerospace systems has been w'idely 
recognized. The industry has done the best 
job of maintaining related capability with 
improvements in estimation methods and 
giving appropriate priority to the hiring and 
training of qualified analysts. Some parts of 
Government, and National Aeronautics and 
Space Administration (NASA) in particular, 
continue to need major improvements in this 
area. Recently, NASA recognized that its 
cost estimation and analysis capabilities had 
eroded to the point that the ability to provide 
timely, reliable estimates w'as impacting the 
confidence in planning many program 
activities. As a result, this year the Agency 
established a lead role for cost estimation and 
analysis. The Independent Program 
Assessment Office located at the Langley 
Research Center v/as given this responsibility. 

This paper presents the plans for the 
newly established role. Described is how the 
Independent Program Assessment Office, 
working with all NASA Centers, NASA 
Headquarters, other Government agencies, 
and industry, is focused on creating cost 
estimation and analysis as a professional 
discipline that will be recognized equally with 
the technieal disciplines needed to design new 
space and aeronautics activities. Investments 


in selected, new' analysis tools, creating 
advanced training opportunities for analysts, 
and developing career paths for future 
analysts engaged in the discipline are all 
elements of the plan. Plans also include 
increasing the human resources available to 
conduct independent cost analysis of Agency 
programs during their formulation, to improve 
near-term capability to conduct economic 
cost-benefit assessments, to support NASA 
management’s decision process, and to 
provide cost analysis results emphasizing 
“full-cost” and “full-life cycle” 
considerations. 

The Agency cost analysis 
improvement plan has been approved for 
implementation starting this calendar year. 
Adequate financial and human resources are 
being made available to accomplish the goals 
of this important effort, and all indications are 
that NASA’s cost estimation and analysis 
core competencies w'ill be substantially 
improved within the foreseeable future. 

INTRODUCTION 

Everyone knows that the enemy of approval 
and startup of new programs is the cost 
analyst who has accurate estimation tools and 
a desire to deliver an honest estimate of what 
the program will really cost. The estimator is 
the person who develops program costs that 
exceeds budget availability, makes the 


* Director, Independent Program Assessment Office 

* Deputy Director, Independent Program Assessment Office 

* Senior Cost Analyst, Independent Program Assessment Office 

Copyright ® 2000 by the American Institute of Aeronautics and Astronautics, Inc. No copyright is asserted in the 
United States under Title 17, U.S. Code. The U.S. Government has a royalty-free license to exercise all rights under the 
copyright claimed herein for Governmental purposes. All other rights are reserved by the copyright owner. 



program less competitive with other programs 
competing for limited resources, and often 
forces the program to focus on objectives 
somewhat less than desirable. Right? No, 
that is not exactly the way good management 
is supposed to work. A responsible program 
or project manager should consider a reliable 
cost estimate as a resource available for 
assuring management success. 

Included in the primary reasons why 
many of today's aerospace program managers 
eventually find themselves in trouble are: 

1 ) Before program approval, they may 
not have adequately defined (systems 
engineered) the system they planned to 
develop and operate. This includes 
developing a full understanding of technical 
and programmatic risks that can be barriers to 
success, 

2) They may not have developed a 
reliable estimate of what it will cost to 
successfully complete the program, and 

3) They may accept the job of 
managing the program with a budget (and 
maybe schedule) with little or no relation to 
the expected actual cost of the program. 

So, what's so difficult? Why can’t 
responsible program management correct 
these deficiencies and more often than not, 
deliver the program product within budget 
and on schedule? A primary requirement for 
success is the program manager wanting to be 
“responsible,” and a second requirement is the 
program manager having the resources to 
complete quality, up-front systems 
engineering and to secure a reliable cost 
estimate. 

This paper will not attempt to address 
the adequacy of systems engineering tools or 
other resources needed by the program 
manager. Much has been recently written 
about efforts to improve tools used in systems 
engineering, both in the United States and 


other countries. Of particular interest is the 
Intelligent Synthesis Environment Program of 
NASA Langley (reported at last year’s lAF 
Congress,)’ with objectives to advance the 
state-of-the-art in near- and far-term 
analysis/design tools and promote 
collaborative engineering among engineering 
organizations. Providing reliable cost 
estimates (the focus of this paper) is a subject 
that has received less attention. Reliable cost 
estimation, as a resource to the program 
manager, has become a scarce commodity, at 
least in NASA. In addition, the image of the 
cost estimator, as the enemy to program 
approval, is wrong. Hopefully, what follows 
will show that NASA recognizes the value 
that reliable cost estimation brings to the 
program formulation and approval process. 

Before turning to how' NASA is 
correcting deficiencies in its program cost 
estimation capabilities, it may be helpful to 
define “cost estimation” as used in the 
following discussion. For the purpose of this 
definition, and for other discussions in this 
paper, “program” refers to an activity 
involving the development and operation ot a 
hardw'are system, or more specifically 
perhaps, a space system. Cost estimation is 
the process of analyzing each hardware 
element, the buildup, integration and test of 
these elements, and the operation of the 
system over some specified life-cycle 
(including disposal of the asset), with respect 
to the cost associated w'ith the total effort. 
The analysis uses analysis tools, or models, 
that relate hardware elements, complexity, 
and risks of failure to expected costs - a 
parametric analysis. Parametric estimation 
involves the development and utilization of 
cost estimation relationships between 
historical costs and program, physical, and 
performance characteristics. The basic 
premise is that the cost of a system, such as a 
spacecraft, is related in an approximate, but 
quantifiable way, to a physical characteristic 
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such as weight, pointing accuracy, number of 
parts, or other attribute. 

There may be at least two different 
types of cost estimates, an “'advocacy” 
estimate and an ‘“independent” estimate. An 
advocacy estimate may be derived by 
program management, and as such, may be 
skewed in ways beneficial to successful 
program advocacy. An independent cost 
estimate is derived by one disassociated with 
the program, and therefore, not encumbered 
by the pressures of advocacy and free to be 
objective - “tell it like it is.” Further, the cost 
estimation and analysis (CEA) competency is 
defined as the total capability of an 
organization to provide the cost estimates 
required by the organization for budget 
planning and execution, and program 
planning and approval. 

BACKGROUND 

NASA, at one time, maintained a 
respectable CEA competency. Qualified cost 
analysts were employed across the NASA 
Centers with appropriate skills and in 
numbers to support the Agency’s needs. 
Several related situations resulted in the 
Agency losing much of its recognized 
competency: 

1) The Agency never recognized CEA 
as a discipline as important as other 
professional disciplines needed for systems 
engineering and development. There was 
never a “career path” available to those 
responsible for CEA, so possibilities for 
career advancement were always in doubt. 
As a result, often other career opportunities 
looked more promising and qualified analysts 
left the work area. 

2) Declining budgets, increasing 
competition for limited funds, and other 
institutional considerations, tended to lead 
many parts of the Agency to underestimate 
program development and operations costs. It 


became obvious that estimators were too 
involved in the advocacy of the programs. 
The appearance of objectivity in the CEA 
process was in question. 

3) Declining workforce led to the 
establishment of hiring priorities that limited 
the replenishment of CEA talent. In addition, 
limited budgets forced drastic reductions in 
investments that would lead to upgrading the 
Agency’s CEA tools and state-of-the-art 
analysis capabilities. 

In the early 1990s, several outside 
advisory groups began to recognize the 
declined state of the NASA CEA competency. 
Of particular importance was the Presidential 
Commission on the Future of Space‘, more 
commonly known as the Augustine 
Committee. The Commission recommended 
to NASA that “an exceptionally well- 
qualified, independent cost analysis group be 
attached to headquarters with ultimate 
responsibility for all top-level cost estimating, 
including cost estimates provided outside of 
NASA.” . As a complementary 
recommendation to foster the independent 
assessment of new proposed programs of 
NASA, the Commission also recommended 
the establishment of an independent ““systems 
concept and analysis group reporting to the 
Administrator.” 

It was in the 1 993 time period that the 
Agency formed a cost analysis group in the 
Headquarters Office of the Chief Financial 
Officer. This group was also given the 
responsibility to organize and conduct 
Independent Annual Reviews (lAR) of the 
progress of Agency programs and report 
findings to the Agency Program Management 
Council. This action recognized that 
independent cost analysis and the 
development of dependable estimates were of 
critical importance to NASA. About 4 years 
later, at the insistence of the NASA Advisory 
Committee, the Independent Program 
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Assessment Office (IPAO) was formed at the 
NASA Langley Research Center. The IPAO 
provides Independent Assessments (lA) of 
new programs, and reports findings and 
recommendations related to the approval of 
programs to the Office of the Administrator. 
The responsibility for the Independent Annual 
Reviews was transferred to the IPAO at that 
time. With the formation of the IPAO, the 
cost estimation function previously 
established all but disappeared, except for a 
small number of cost analysts that joined the 
IPAO at Langley to continue cost analysis to 
support the review and assessment activities. 

During the period from the mid-1980s 
to the present, for the reasons stated earlier, 
there was a greater than 50 percent attrition in 
the NASA CEA competency. At present, the 
Agency of over 18,000 people employs less 
than 25 full-time, career cost analysts. In 
addition to the inability to develop sufficient 
cost information for management of its 
programs, there is concern for the Agency’s 
ability to adhere to Federal standards in this 
area. The various remaining elements of the 
cost community are decentralized and have 
ambiguous roles and responsibilities. There is 
ineffective use and limited sharing of these 
resources, and there is no clear definition of a 
professional cost estimation competency. 

The Agency currently conducts 
systems engineering, budget development, 
and cost estimating separately. There is 
inconsistent use of cost estimating techniques 
and tools, and there are outdated tools and 
cost databases. During program planning, 
program requirements and risks to program 
success are not adequately defined, and 
usually, no program life-cycle costs are 
considered in the approval process. Cost 
analysis tools are insufficient for estimating 
program cost for programs that are to be 
conducted with “new ways of doing business” 
or “faster, better, cheaper.” And to add to 


these deficiencies, budget-driven processes 
often are unsupported by CEA. 

The Federal Office of Management 
and Budget (0MB) is pushing for improved 
cost analysis prior to budget submission. The 
0MB goals include better schedule and 
performance goals, more realistic baseline 
cost, the inclusion of independent cost 
estimates, and full life-cycle cost, cost benefit, 
and estimate of risk and uncertainty. All 
these cost considerations should be used as a 
basis for selecting future NASA programs. 

Lastly, in recognition of the above 
deficiencies, the Administrator recently 
directed improvements in NASA’s 
independent cost estimation abilities. His 
direction resulted in the assignment to the 
IPAO this year the Agency lead responsibility 
to correct the deficiencies and restore the 
Agency’s cost estimation and analysis 
competency. The following discussion details 
the plans to fulfill the new IPAO role. 

OVERALL STRATEGY 

With the assignment of the lead role 
for cost estimation and analysis to the IPAO, 
the primary' responsibility of the Office will 
be the reconstitution of a comprehensive, core 
CEA competency for the Agency. This 
responsibility includes the development of a 
CEA strategic plan for the accomplishment of 
the role, providing leadership for the 
establishment of an adequately staffed and 
skilled NASA cost estimating community, 
and rallying the community in a coordinated 
effort to eliminate the cost estimation 
deficiencies currently plaguing the Agency. 
The IPAO, NASA Headquarters, and the 
NASA Centers will work together toward 
implementation of state-of-the-art costing 
capabilities including investing in new and 
upgraded tools, and organizing appropriate 
training. The Office will function as an 
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independent Agency resource for program, 
Center, and Enterprise management by 
providing independent cost estimates to 
support program planning and 
implementation. 

GOALS 

The overall goal of the lead CEA 
activity is to prepare NASA for the future by 
restoring the ability to develop accurate, 
reliable cost estimates of the Agency's 
programs and provide confidence for senior 
leadership that NASA's programs are based 
on a solid foundation of understanding cost 
and risk. The new CEA core competency 
created will involve state-of-the-art business 
practices for a full spectrum of cost analysis 
tools and processes for managers and assure 
integration of cost estimating, systems 
engineering and management, and budget 
development. Cost estimates will be timely, 
and a better understanding of risk and 
schedule issues will be developed prior to 
program approval. 

Specific examples of CEA 
competency improvement goals include 
making the following changes in the way 
NASA derives its program cost information: 

1. Traditional, non-integrated tools will be 
replaced by state-of-the-art, integrated tools. 

2. Center oriented cost organizations will 
become part of an intra-Agency cost 
community. 

3. Functionally focused cost tasks will 
transition to integrated product teams that 
include estimating professionals. 

4. Non-professional '‘journeymen” cost 
analysts will be trained and matured into core 
competency cost analysts. 

5. Non-full cost estimates will be substituted 
with full cost estimates. 

6. Center oriented tools will become shared 
Agency tools. 


7. Separate development and operational 
costs estimates will be replaced by integrated, 
full life-cycle estimates. 

The focus of the CEA improvement 
initiative will be on people , tools . 
organization , and processes . 

PEOPLE 

The most important resources in any 
activity are the people involved. For success, 
the people must be competent, motivated, and 
have the other resources (tools, money, etc.) 
to get the job done. As stated earlier. NASA is 
short of properly skilled cost analysis 
workforce. Therefore, attention to the 
“people” factor becomes a first-order priority 
of the NASA CEA improvement initiative. 

Adequate compensation is a genuine 
motivator, but it is well known that when one 
decides to work in the public sector, there are 
limits to what one can expect in terms of pay. 
Therefore, there must be other motivators that 
cause an employee to want to make a career 
in government. With respect to those 
working in the CEA field, for example, 
expectations of being treated as a valued 
employee with definite opportunities for 
professional growth is a true plus. 
Unfortunately, in NASA, there has not existed 
much of a professional career path. In fact, 
while systems development activities require 
a wide variety of professional disciplines to 
implement the program, the activities also 
require CEA, but CEA is not viewed as a 
discipline in the same way as other areas 
(such as the engineering disciplines). A 
primary goal of the initiative is help establish 
CEA as a recognized professional discipline 
with a formal growth ladder and opportunities 
for senior level positions for individuals 
dedicated to the area. The IPAO will work 
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with NASA's human resource organizations 
to help develop this potential. 

The IPAO will assist the NASA 
Centers in recruiting new employees to the 
Agency’s core competency of CEA. A 
database of qualified individuals with interest 
in working for NASA in this area will be 
maintained, and as a Center wishes to 
supplement or replenish its CEA staff, the 
Office will offer appropriate assistance in 
seeking qualified applicants. The Office will 
also work with the Agency’s career training 
organizations to help develop CEA-related 
training opportunities for the staff. This 
should include various professional and 
leadership training as well as specific training 
in the CEA discipline to improve the 
individual’s skills. In addition, the IPAO will 
help develop Agency-wide on-the-job training 
that will improve professional development 
while creating opportunities for inter-Center 
exposure and cross-fertilization of CEA 
methods among Centers. 

Lastly, the IPAO will invest in the 
development of a college undergraduate 
course on CEA. It is interesting that in an 
environment of economic pressure, constant 
push to reduce cost, and to build systems 
“faster, better, and cheaper,’’ we graduate 
engineers without a notion of how much it 
costs to build the systems they are trained to 
design. The course will be designed with the 
objective of introducing college engineering 
students to cost analysis, and will be offered 
to any engineering school desiring to improve 
its engineering curriculum in this regard. If 
an appropriate one can not be found, there is 
some thought currently being given to the 
development of an undergraduate-level text 
on the subject of CEA for the purpose of 
aiding in the teaching of the course. 


TOOLS 

NASA must be capable of generating 
responsive, reliable, quality cost estimates of 
future missions (such as Mars Exploration) 
that involve the use of new’ technologies and 
innovative approaches or concepts for 
satisfying mission objectives. In order to 
achieve this capability, the IPAO will be 
responsible for advancing the state-of-the-art 
in cost models and analytical tools. One of 
the first capabilities identified for immediate 
development is the integration of systems 
development cost models with operations cost 
models. Efforts are underway to integrate the 
Space Operations Cost Model (SOCM)^, 
which is an internally developed model, with 
several commercial models such as the 
NASA/Air Force Cost Model (NAFCOM) 
and the PRICE Cost Model. 

There are on-going discussions w ithin 
the CEA community about the requirements 
of the next generation of NASA cost models. 
In the near future, NASA management will 
require “full cost’’ estimates, estimates that 
include the workforce, general & 
administrative costs, facility, and 
program/project costs. Models to estimate 
workforce cost are being developed to meet 
this requirement along with other 
methodologies to estimate the full cost of 
NASA projects. The NASA Integrated 
Financial Management System, currently 
being developed elsewhere in NASA, will 
assist in providing the CEA discipline with 
the full cost accounting data needed to 
develop full cost estimates. Also, models will 
be required to more accurately determine the 
cost impact of new systems development 
approaches such as “faster, better, cheaper.’’ 

The Intelligent Synthesis Environment 
(ISE) program is a NASA initiative to 
develop a virtual reality design environment. 
The goal is an advancement of the simulation 
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based design environment involving the 
integration of design and cost models with 
analytical tools using intelligent systems 
technology. As a result of this new 
environment, the time to develop new system 
designs and to estimate the costs will be 
greatly reduced. IPAO will be collaborating 
w'ith the ISE program, specifically with the 
Cost and Risk Management Technology 
element of ISE, to develop cost and risk 
models that work within the ISE environment. 
These cost models must be capable of 
reflecting the revolutionary reduction in the 
time and cost of various phases of the design 
cycle and be state-of-the-art tools. Among 
the analytical tools planned for development 
is an improved schedule assessment tool. 

IPAO will lead NASA’s participation 
in the establishment of joint tool development 
efforts with other government and private 
industry cost analysis organizations. 
Recently, NASA became a member of the 
newly established Consortium on Space 
Technology Estimating Research (CoSTER) 
organization. The CoSTER includes most of 
the government agencies with an interest in 
space. This relationship will likely result in 
joint tool development investments that will 
benefit a broader range of government 
organizations. In addition, there will be 
efforts to assure cost model training 
opportunities are made available to all cost 
analysts. The result of CEA tool investments 
will be powerful cost-, schedule-, and risk- 
estimating systems that help NASA better 
understand the cost of doing business and 
make the right management decisions. 

ORGANIZATION 

The IPAO will carry out its CEA 
responsibilities with active participation of all 
NASA Centers and Headquarters. The 
Agency-wide initiative includes the creation 
of a CEA Steering Group with members 


representing all the cost estimation 
organizations across NASA. This group is 
actively involved in establishing overall goals 
of the initiative, in decisions affecting the 
future of the CEA competency, in defining 
workforce and analysis tool requirements, and 
in the implementation of the initiative’s 
elements. Group members represent the 
CEA-related interests of their home Centers, 
serve to share experiences (or lessons- 
learned) from cost analysis activities, and 
accept complementary responsibilities lor 
various initiative actions. In addition, the 
group will facilitate an Agency-oriented CEA 
culture rather than a specific Center-oriented 
culture. 

The IPAO, in cooperation with the 
CEA Steering Group, is organized to serve 
the Agency in several important ways. It is a 
primary interface with other government 
agencies to coordinate inter-Agency CEA 
activities, perform completely independent, 
non-advocate cost estimates in support of 
program formulation, and provide other CEA 
support to Centers when Center resources are 
insufficient. For the CEA initiative, the IPAO 
workforce was increased by eight, and 
sufficient funds have been provided to secure 
contracted CEA analysis to meet support 
demands. In addition to the IPAO, it is 
expected that the Centers’ CEA organizations 
will also be appropriately expanded to meet 
the cost analysis requirements unique to each 
Center. 

The IPAO CEA responsibilities 
include serving as the voice of the Agency’s 
cost estimation and analysis community. The 
Office will integrate the fragmented concerns 
of 10 NASA Centers into focused CEA 
community concerns and issues, and this 
consolidation will enable the community s 
voice to be much stronger than each Center 
acting separately. It is expected that this will 
result in more positive results in efforts to 
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resolve the overall CEA competency 
deficiencies. 

PROCESSES 

Since one of the responsibilities of the 
IPAO in the CEA initiative is to oversee the 
quality control of the Agency's CEA 
products, the Office will be working to create 
standards for the discipline and to have all 
Center ‘’buy-in” to the standards. The main 
focus on creating standards will be to ensure 
that CEA processes are consistent and 
conform to the best business practices, 
provide timely and accurate cost estimates, 
and are of maximum value to the Agency. 
The CEA Steering Group will lead in the 
development of the standards, and it is 
expected that all processes will be subject to 
ISO certification. Included in the processes 
will be guidelines for cost analysts' continued 
participation through the project's life and 
being involved in the creation of full-cost 
estimates of the system development, 
operation, and disposal. A closer relationship 
between those responsible for developing 
systems requirements and those responsible 
for estimating requirement's cost must be 
developed. The full integration of CEA into 
design activities is necessary. Cost analysts 
must be active participants in initial system 
trade studies that should be conducted to 
derive the system approach that achieves the 
requirements in the most cost-effective way. 
In today’s program management world, the 
cost analyst must remain an active, but 
objective member of the team throughout the 
entire life-cycle of the program. 

SUMMARY 

The IPAO led CEA initiative will 
focus on up-front planning and continued 
improvement of the NASA CEA competency 
and will provide key benefits to the future 


systems development activities of the 
Agency. These benefits include the 
following: 

1 ) A new cost estimating culture and 
an integrated cost community that 
better serves the Agency. 

2 ) Enhanced costing skill and a 
professional career path for 
analysts. 

3 ) Better definition of systems 
development and operational risks 
and an estimate of the costs to 
mitigate these risks. 

4) Reliable, responsive, full life-cycle 
cost estimates. 

5) Cost estimate quality control. 

6) Continual CEA support throughout 
project life-cycle. 

7 ) Databases for improved cost 
estimating and cross program 
analysis. 

8) State-of-the-art, user friendly tools 
and processes that accommodate 
NASA’s new ways of conducting 
systems development activities. 

9) Make the Agency more 0MB- 
compliant. 

10) Better fiscal support and budget 
defense 

The IPAO, in its lead role for CEA, 
will function as an independent Agency-level 
resource and will ensure all Centers are 
involved in the reconstitution of this most 
important Agency discipline. The bottom line 
is that after these changes are instituted, both 
the Agency and those in charge of NASA’s 
appropriations will have a much greater 
confidence that our proposed costs will also 
be our actual costs. This will make our 
budgets more defensible, leading to better 
Congressional support. 


8 


REFERENCES 


1. Monell, Donald W. and Piland. William 
M., “Aerospace Systems Design in NASA's 
Collaborative Engineering Environment," 
lAF Paper 99. U.l .01, October 1999. 

2. Augustine. N., “Report of the Advisory 
Committee on the Future of the U.S. Space 
Program,” 1990. 

3. Moore, Arlene, Proceedings of the 
International Society of Parametric Analysts, 
“Parametric Prospectives in the Information 
Age,” New Orleans, May 1997. 


9 



